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A New Dell ign for Evaluation in Mathematics Education: 
Longitudin&l Comprehensive Achievement Monitoring 



by 

William P. Gorth 
and 

Dwight W. Allen 



The authors will describe a new design for longitudinal comprehen- 
sive achievement monitoring. The design has been developed 
under the sponsorship of a four-year grant from the Charles F. Kettering 
Foundation. It has been operating in five high school mathematics 
courses for almost two years. The components of the design which dis- 
tinguish it from the usual classroom achievement testing are: (a) a model 

of school learning to direct the choice of variables to measure, (b) a 
comprehensive set of course objectives defined in behavioral terms, (c) a 
longitudinal scheduling of testing, and (d) a complete system to process 
data and report results to the teacher and individual students. The usual 
classroom achievement testing does not have a model of school learning, 
schedules tests for immediately after instructional treatment, uses a 
narrow set of generally poorly defined objectives and makes only a crude 
analysis of results. 

The model . Several recent researchers have attempted to model 
school learning. A semi-quantitative model (Carroll, 1967) appears par- 
ticularly useful as a starting point in choosing variables to measure and 
adequately describe school learning, but should be expanded. 



verance, which Carroll uses. Second, although a studei^js^^ be motivated 
to perform a particular task, his anxiety at entepisig the test situation 
or being evaluated mav interfere with severalTof the skills which are 



necessary for perfori :ect of anxiety on complex 



learning has been in {er and will be included in our 



Our model incorporates student and instructional variables. Firsx 
our model includes a measure of student motivation rather than of-^rse- 
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model. Third, sociological variables are included and appear parti- 
cularly important in public schools which enroll students from a broad 
range of family backgrounds. Fourth, the variables of rate of learning 
and initial achievement correlate very highly with achievement output, 
but have been found in preliminary studies to be independent of one 
another. Finally, the instructional variables include measures of the 
teachers, treatment characteristics, and time. 

Comprehensive objectives . It has only been in recent years that 
teachers have begun to define their objectives in behavioral terms. 
Authors like Mager have written persuasively in favor of specifying in- 
structional objectives in terms of observable student behaviors. The 
teacher is expected to define the objectives of his entire course before 
it begins in terms of observable student behavior by conqposing questions 
which measure acceptable levels of student performance for the objectives 
he set for his course. 

Longitudinal testing schedule . The third distinguishing aspect 
of the comprehensive achievement monitoring is the longitudinal consi- 
deration of students* achievement on each course objective. In usual 
classroom testing a test of achievement is given immediately after an ob- 
jective or set of objectives has been presented by the teacher. The test 
usually includes items which measure only the objectives taught since the 
last testing. Therefore, the teacher has available estimates of student 
achievement only on the objectives he has Just taught. The usual 
situation contrasts with that of comprehensive achievement monitoring 
where an estimate of student achievement on each of the objectives is 
available at each testing period. Therefore, teachers can make state- 
ments about a student's pre-instruction, post- instruct ion, and retention 
of material as well as rate of learning. For example, if objectives 
one through ten are taught consecutively, and achievement is monitored 
after each is taught, then at time four. Just after objective four has 
been taught, the estimate of student achievement on objective four is 
an Immediate post- treatment achievement. The estimates made at the same 
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time for objectives one through three are retention measures and those 
of objectives five through ten are pre* instruct ion measures. 

The system . The system for achievement monitoring includes the 
parts (a) model of the parameters, (b) focus of the evaluation, (c) re- 
sources for the evaluation, (d) collection of the data, (e) organisation 
of the data, (f) analysis of the data, and (g) report of the analysis. 
Computer programs have been developed to handle the large tasks of 
analysis and reporting results. Other coiq>onents of the system have 
been designed to operate at several levels, i.e., both with or without 
the computer. 

Implications . The new design for evaluation in mathematics 
education provides longitudinal, comprehensive information on the 
achievement of individual students and groups of students on the perfor- 
mance objectives of the course. It is able to provide information on 
achievement on a standard scale for the course across time (a) before 
instruction, (b) immediately after instruction, and (c) a long time after 
instruction. 

The data collected by the system could be used in several ways. 
First, course improvements could be made based upon the model of school 
learning. High entry achievement, low post- instruct ion achievement, or 
forgetting by the students in a course would each Justify modifications 
in the course. Second, individual student school learning patterns could 
be plotted and decreases, plateaus or sharp rises in achievement would 
suggest different activities for the students. Finally, if alternate 
instructional treatments were administered, each to part of the class, 
the achievement patterns of the parts would display any differences in 
achievement between them immediately after or several weeks after the 
treatment. Therefore the system could be used in enq>irical research in 
teaching. 



